Enhancing Automated Network Management

Network management benefits from automated tools. With the recent advent of software-defined principles, automated tools have been proposed from both industry and academia to fulfill function components in the network management control loop. While automation aims to accommodate the ever increasing network diversity and dynamics with improved reliability and management efficiency, it also brings new concerns as it’s becoming more difficult to understand the control of the network and operators cannot rely on traditional troubleshooting tools. Meanwhile, how to effectively integrate new automation tools with existing legacy networks remains a question. This dissertationpresents efficient methods to address key functionalities within the control loop in the adaption of automated network management.Identifying the network-wide forwarding behaviors of a packet is essential for many network management tasks, including policy enforcement, rule verification, and fault localization. We start by presenting AP Classifier. AP Classifier was developed based on the concept of atomic predicates which can be used to characterize the forwarding behaviors of packets. There is an increasing trend that enterprises outsource their Network Function (NF) processing to a cloud to lower cost and ease management. To avoid threats to the enterprise’s private information, we propose SICS based on AP Classifier, a secure and dynamic NF outsourcing framework. Stateful NFs have become essential parts of modern networks, increasing the complexity in network management. A major step in network automation is to automatically translate high level network intents into low level configurations. To ensure those configurations and the states generated by automation match intents, we present Epinoia, a network intent checker for stateful networks. While the concept of auto-translation sounds promising, operators may not know what intents should be. To close the control loop, we present AutoInfer to automatically infer intents of running networks, which helps operators understand the network runtime states

Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers

We propose to address quadrupedal locomotion tasks using Reinforcement Learning (RL) with a Transformer-based model that learns to combine proprioceptive information and high-dimensional depth sensor inputs. While learning-based locomotion has made great advances using RL, most methods still rely on domain randomization for training blind agents that generalize to challenging terrains. Our key insight is that proprioceptive states only offer contact measurements for immediate reaction, whereas an agent equipped with visual sensory observations can learn to proactively maneuver environments with obstacles and uneven terrain by anticipating changes in the environment many steps ahead. In this paper, we introduce LocoTransformer, an end-to-end RL method for quadrupedal locomotion that leverages a Transformer-based model for fusing proprioceptive states and visual observations. We evaluate our method in challenging simulated environments with different obstacles and uneven terrain. We show that our method obtains significant improvements over policies with only proprioceptive state inputs, and that Transformer-based models further improve generalization across environments. Our project page with videos is at https://RchalYang.github.io/LocoTransformer .Comment: Our project page with videos is at https://RchalYang.github.io/LocoTransforme

CEIL: Generalized Contextual Imitation Learning

In this paper, we present \textbf{C}ont\textbf{E}xtual \textbf{I}mitation \textbf{L}earning~(CEIL), a general and broadly applicable algorithm for imitation learning (IL). Inspired by the formulation of hindsight information matching, we derive CEIL by explicitly learning a hindsight embedding function together with a contextual policy using the hindsight embeddings. To achieve the expert matching objective for IL, we advocate for optimizing a contextual variable such that it biases the contextual policy towards mimicking expert behaviors. Beyond the typical learning from demonstrations (LfD) setting, CEIL is a generalist that can be effectively applied to multiple settings including: 1)~learning from observations (LfO), 2)~offline IL, 3)~cross-domain IL (mismatched experts), and 4) one-shot IL settings. Empirically, we evaluate CEIL on the popular MuJoCo tasks (online) and the D4RL dataset (offline). Compared to prior state-of-the-art baselines, we show that CEIL is more sample-efficient in most online IL tasks and achieves better or competitive performances in offline tasks.Comment: NeurIPS 202